Gradient

Overview

For a function of \(n\) variables, \(f(x_1, x_2, \ldots, x_n)\), the gradient \(\nabla f\) packages how sensitive \(f\) is to tiny moves in \(x_1, x_2, \ldots, x_n\). It points in the direction of steepest ascent (where \(f\) increases fastest, per unit step in the plane), and its length is that rate of increase.

Key idea

For the sake of simplicity, let’s focus on a function of two variables, \(f(x,y)\). The partial derivatives \(\partial f/\partial x\) and \(\partial f/\partial y\) tell you how \(f\) changes when you nudge only \(x\) or only \(y\). Putting them in a vector \(\nabla f = \bigl(\partial f/\partial x,\; \partial f/\partial y\bigr)\) gives you the single direction in the plane that matches the overall slope of the surface \(z = f(x,y)\).

In the interactive demo above, \(f(x,y) = 1 - x^2 - y^2\) is a downward-opening “cap” with its maximum at the origin. Drag the point and watch how \(\nabla f\) always points toward that peak (inward, perpendicular to the circular contours).

Mathematical formulation

For \(f(x,y) = 1 - x^2 - y^2\), the gradient is:

\[ \nabla f(x,y) = \left(\frac{\partial f}{\partial x},\,\frac{\partial f}{\partial y}\right) = (-2x,\,-2y). \]

So on the \(x\)-axis to the right of the origin, \(\partial f/\partial x < 0\) (moving right lowers \(f\)); above the origin on the \(y\)-axis, \(\partial f/\partial y < 0\) (moving up lowers \(f\)). At \((0,0)\), both partials vanish: \(\nabla f = \mathbf{0}\) — a critical point (here, the global maximum).

Level set \(f = c\) is \(x^2 + y^2 = 1 - c\) (a circle of radius \(\sqrt{1-c}\) when \(c \le 1\)). The gradient is always radial, toward the center — orthogonal to those circles, as it must be for steepest ascent to cross contours at right angles.

Relation to contours

Walking tangent to a contour keeps \(f\) constant to first order, so directional derivative along the tangent is zero — equivalently, \(\nabla f\) is normal to the contour. That is why, in the picture, the orange vectors pierce the blue circles at \(90^\circ\).

What you’re seeing

The blue curves are level sets \(f(x,y) = c\) (constant height) or contours. For this \(f\), they are circles \(x^2 + y^2 = 1 - c\). The orange arrow is \(\nabla f = (-2x,\,-2y)\) which is perpendicular to the contour through that point and aimed uphill.